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This work extends the results of the recently developed theory of a rather complete thermody- 
namic formalism for discrete-state, continuous-time Markov processes with and without detailed 
balance. We aim at investigating the question that whether and how the thermodynamic structure 
is invariant in a multiscale stochastic system. That is, whether the relations between thermody- 
^ ■ namic functions of state and process variables remain unchanged when the system is viewed at 

o : 

O . different time scales and resolutions. Our results show that the dynamics on a fast time scale con- 

, tribute an entropic term to the "internal energy function", us{x), for the slow dynamics. Based on 

■ the conditional free energy us{x), one can then treat the slow dynamics as if the fast dynamics is 

^ ! 

ly^ . nonexistent. Furthermore, we show that the free energy, which characterizes the spontaneous orga- 

I nization in a system without detailed balance, is invariant with or without the fast dynamics: The 

' fast dynamics is assumed to reach stationarity instantaneously on the slow time scale; they have 

no effect on the system's free energy. The same can not be said for the entropy and the internal 
^ , energy, both of which contain the same contribution from the fast dynamics. We also investigate 

I the consequences of time-scale separation in connection to the concepts of quasi-stationaryty and 

steady-adiabaticity introduced in the phenomenological steady-state thermodynamics. 
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I. INTRODUCTION 



Stochastic dynamics that can be described by a Markov process embody a rich_t 
namic structure. Recently, inspired by the discovery of the fluctuation theorem 
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lermody- 
-7|, there 



is a growing interesting 



in concepts such as Gibbs entropy and free energy associated with 



Markov processes [8l4l3l|. The free energy of a stochastic dynamics is intimately related to 
the relatij;e entropy which has found great importance in the general theory of dynamical 
systems 
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16|. 



In the very recent paper [10| , a rather complete thermodynamics has been presented for 
discrete-state, continuous-time stochastic Markov systems with or without detailed balance. 
The thermodynamics are characterized by: 

(i) A balance equation for the Gibbs entropy that includes a non-negative entropy pro- 
duction rate a. 

(ii) A decreasing free energy dF/dt < 0. 

(iii) A decomposion of a into —dF/dt and the house keeping heat Qhk] both being non- 
negative. 

Assertion (iii) indicates that the total irreversibility has two distinct origins: the spontaneous 
self-organization into a nonequilibrium steady state, and the continuos environmental drive 
that keeps the system away from its equilibrium. These terms respectively correspond 
to the Boltzmann's thesis and the Prigogine's thesis 10|. For systems in a non-driving 
environment, detailed balance holds. Then Qh^ = 0, a = —dF/dt, and the system relaxes 
to an equilibrium steady state with cr = 0. The mathematical theory is an abstraction for 
an earlier phenomenological study of nonequilibrium steady state thermodynamics by Oono 



and Paniconi 



17|. 



For almost all applications of stochastic dynamic theories in physics, chemistry and biol- 
ogy, there will be multiple time scales, and often with a significant separation. Recall that 
in the Gibbs formalism for equilibrium statistical mechanics, the conditional free energy, 
plays a central role in applications: one usually does not work with the pure mechanical en- 
ergy of a system; rather, one works with a conditional free energy from coarse-graining and 
develops a partition function thereafter. The present work focuses on this important issue: 
Whether the new thermodynamic structure is invariant in a multiscale stochastic dynamical 



system. Or in other words: whether the relation between state and process variables remains 
unchanged when the system is viewed at different time scales. 

We show that the dynamics on a fast time scale contribute an entropic term to the 
"internal energy function", Us{x), for the slow dynamics. Us{x) should be understood as 
the conditional free energy. And based on ^^(x), one can then treat the dynamics on the slow 
time scale as if the fast dynamics is nonexistent. Futhermore, we show that the free energy 
(which characterizes the spontaneous organization in a system) is invariant with or without 
the fast dynamics. Since the dynamics on the fast time scale reaches their stationarity 
instantaneously on the slow time scale, they have no effect on the system's free energy. 
The same can not be said for the entropy and the internal energy, both of which contain the 
same entropic contribution from the fast dynamics. Since "free energy equals internal energy 
minus entropy", there is a compensation. Finally, we study how the time-scale separation 
affects the concepts of stationary and steady-adiabatic processes introduced by Oono and 



Paniconi 17|. 



II. ADIABATIC APPROXIMATION 



In this section we follow the ideas of quasi-steady state approximation, or singular per- 



turbation 
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21|. in stochastic dynamics to perform an adiabatic treatment in a generic 
Markovian stochastic process where separation of time scales is possible. We discover that 
the concept of conditional probability is a very natural language for performing this multiple 
time scale analysis. 

Consider a Markov system whose state can be represented by a dual vector {x,y), where 
variables x and y take discrete values. Let p{x,y) be the probability of state {x,y) and 
v{x,y; x' ,y') be the transition probability per unit time from state {x,y) to state {x',y'). 
We further assume that v{x,y; x,y') ^ v{x" ,y"; x'" ,y"') for all y ^ y', x" ^ x'", and 
{x",y") 7^ {x"',y"'). That is for any given x, the transition y ^ y' is much faster than all 
transitions involving changing x" — )■ x'". If the sets of all possible values attained by x and 
y are finite, the master equation (or forward Kolmogorov equation) for this system can be 
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written as 

^^^J^' = ^P{x,y')v{x,y';x,y) - p{x,y)v{x,y;x,y') 

all y' 

+ '^^P{x',y')v{x',y';x,y) -p{x,y)v{x,y;x',y'). (1) 

x'j^x all y' 

By adding the above equation over all values of y and taking into consideration that p{x) = 
JZyPi^^y) obtain 

= ^ P{x, y')v{x, y'; X, y) - p{x, y)v{x, y; x, y') 

all y,y' 

x'^x all y,y' 

Note that the first summand in the right hand side of the above equation equals zero because 
each term in it is added and subtracted once. After some algebra this equation can be 
rewritten as 

^ = 5^p(x')T(a;'; x) - p{x)T{x; x'), (2) 

x' 

with 

T{x;x') = ^p{y\x)v{x,y;x',y'). (3) 

y,y' 

In the previous equation p{y\x) is the conditional probability defined as 

p{y\x) = p{x,y)/p{x). (4) 

In order to find the equation governing the dynamics of p{y\x) let us differentiate (jlj) to 
obtain the following expression after some algebraic steps: 

.dp{y\x) dp{x,y) .dp{x) 

pi^) — — = — — piy\^) 



dt dt dt 

Further substitution of ([1]) and ([2]) into this equation leads to 
dp{ii\x 



dt 



^p{y'\x)v{x, y'\ X, y) - p{y\x)v{x, y; x, y'), (5) 



where we have neglected all terms multiplied by either v{x,y; x' ,y') [x 7^ x' and {x,y) 7^ 
{x',y')] or T{x;x'), based on the fact that they are much smaller than v{x,y'; x,y). 
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Finally, from the same assumed time-scale separation, we can make an adiabatic approx- 
imation and suppose that p{y\x) ~ P^{y\x), where the conditional stationary distribution 
satisfies 

^p'{y'\x)v{x,y';x,y) - p'{y\x)v{x,y;x,y') = 0. (6) 
y' 

In summary, after performing the above described adiabatic approximation, the dynamics 
of p{x) are governed by ([2]), where the effective transition probability from state x to state 
x' is given by 

T(x; x') = ^p'{y\x)v{x, y; x', y'), (7) 

y,y' 

while p'^{y\x) is the solution of ([6]). Notice that the adiabatic approximation that we have 
introduced in the above paragraphs is equivalent to that introduced by Pigolotti and Vulpiani 

Q- 

III. THERMODYNAMIC STATE FUNCTIONS 
A. Internal energy 

Consider a molecular system that is irreducible; and thus has a unique long-time sta- 
tionary probability distribution p'^{x,y). Further assume that the system is in contact with 
an isothermal bath with chemical potential difference. Thus, we can define, following Ge 
and Qian , the energy function associated to state (x, y) via the stationary distribution 

p^{x, y) as 

u{,x, y) = -ksT \ogp'{x, y), (8) 

where /c^ is the Boltzmann constant and T is the absolute temperature. In systems with 
detailed balance p'^{x,y) equals the thermodynamic-equilibrium probability distribution 
p'^{x,y) and Eqn. ([8]) is equivalent to Gibbs' grand canonical ensemble. When detailed 
balance is not fulfilled, the above definition of internal energy is related to the stochastic 



potential studied by Kubo et al. 22 



From (IH]), the mean internal energy of the mesoscopic state p{x, y) can be written as 

U = ^pix,y)u{x,y) = -kBT^p{x,y)\ogp'{x,y). (9) 



x,y x,y 
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By substituting (jl]) into ([9]) this last equation can be rearranged as follows: 

U = ^p{x,y)u{x,y) = ^p{x){usix) + (10) 

x,y X 

where 

us{x) = —ksTlogp^i^x) and uf{x) = —kBT'^^p{y\x)\ogp'^{y\x). (11) 

y 

Moreover, if we impose the adiabatic approximation stating that p{y\x) ~ P^{y\x), 

uf{x) = -kBTY,f{y\x)^ogf{y\x). (12) 

y 

These results imply that the internal energy can be split in two components {U = Us + Up) 
corresponding to the slow {Us = J2xP(^)'^s{x)) and fast {Up = Y1ixP^^)'^f^-'^)) scales, 
respectively. 

B. Entropy 

The Gibbs entropy is defined as usual: 

S = -kB^p{x,y)\ogp{x,y). (13) 

x,y 

Substitution of (jl]) into (fT3l) leads to 

S = -k B^p{x) log p(x) - kB^p{x)^p{y\x) log p{y\x). (14) 

X X y 

We see that, once more, the entropy can be separated into slow and fast components [S = 
Ss + Sp) respectively defined as 

Ss = -kB^^pix) log p{x) and Sp = -kB^p{x)'^p{y\x)logp{y\x). (15) 

X X y 

If we enforce the adiabatic approximation {p{y\x) = p'^{y\x)), the fast component becomes 
= J2xPi^)^Fix), with 

sp{x) = -kB^p'{y\x) logp'{y\x) (16) 
y 

We note by comparing Eqns. (fT2l) and (fT6|) that up{x) = Tsp{x) due to the adiabatic 
approximation. 
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C. Free energy 



From its definition, F = U — TS, and Eqns. ([9]) and (fT3|) . the Helmoltz free energy is 
given by [loj|: 

p{x,y) 



F = keT p{x,y) log ( - 

r1. \i 



X y 

In tliis case it is also possible to identify slow (F5) and fast (Fp = 'Yl,xP{^)fF{^)) components 
for the free energy, where 

F5 = fcsT Vp(a;)log-^^ and /^(x) = fcj;T Vp(y|x) log ^Jf^. - 

However, the imposition of the adiabatic approximation implies that fpix) = Va;, and so 
that Fp = '^,j.p{x)fF{x) = 0. This agrees with the fact that enforcing the adiabatic approx- 
imation is equivalent to assuming that the fast time-scale distribution {p{y\x)) equilibrates 
instantaneously with the slow one {p{x)) for every given x. Therefore, the system's free 
energy is invariant whether one considers or neglects the faster dynamics, as long as there 
is a reasonable separation of time scales. 



D. Slow- dynamics perspective and whole-system- level interpretation 

First we note from ( !T2|) and ( !T6|) that, once the adiabatic approximation has been made, 
sp{x) = Up{x)/T. This term should be regarded as the entropy of a state x due to the fast 
dynamics of variable y within the given x. Then, (fTOl) indicates that the energy of the slow 
time scale obeys 

us{x) = (^^p'{y\x)u{x, y)j - Tsf{x) = u{x) - Tspix), (18) 

where the first term on the right-hand-side, u{x), is the mean internal energy of state x. 
Finally, in terms of u{x), one has the canonical form of the thermodynamics for the slow 
variable 

Fs = F = '^p{x)us{x) + ksT'^p^x) \ogp{x). (19) 

X X 
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To gain more insight into the physical meaning of us{x) we shall discuss another feasible 
interpretation for this quantity when x is a continuous variable. In such a case, us{x) takes 
the form of a potential of mean force. In fact, noting that p'^{x) = YlyP'^i^yy)^ together 
with the definitions for u{x,y) ([H]) and Us{x) flTT]) . one has 

usix) = -kBTlog'^exp{-u{x,y)/kBT) (20) 

y 

while 

d ^ ^yexp{-u{x,y)/kBT)du{x,y)/dx 
dx ^ V exp(-M(x,|/)//cBT) 



which corresponds to the usual potential of mean force definition 23|. 



IV. TIME EVOLUTION AND THERMODYNAMIC PROCESS FUNCTIONS 
A. Time derivative of the thermodynamic functions 

Following Ge and Qian 



10|, we shall differentiate the expressions for U, S, and F — Eqns. 
(fTOj) . (fT4|) . and (fT7|) — and write the corresponding rates of change in terms of energy and 
entropy fluxes; since understanding these fluxes under different conditions provides valuable 
information regarding the system dynamic and thermodynamic behavior. In particular, 
we are interested in investigating how the slow and fast dynamics subspaces contribute 
to the energy and entropy fluxes, and whether their structure remain invariant from the 
slow-dynamics perspective. 

The time derivatives for for U, S, and F are calculated in Appendix [Aj After imposing 
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the adiabatic approximation p{y\x) ~ j9^(?/|a;) on the corresponding expressions we obtain: 
if = — ^—'^^{p{x')T{x';x)—p{x)T{x;x'))log^^ ^ 



2 z_.-v ' V ' ^ ....... o^,^^,^ 

X] ^{jp\y'\x)v{x, y'] x, - p\y\x)v{x, y; x, y')) log ^I^^T^' (22) 



' - ^(-)^(-' log ^g^, (23) 
p(x')T(x';x) T(x';x) 



S = Y$^(p(x')T(x';a;)-p(x)T(x;x')) (l^S ^yY^f^ " 



Tfx; x' 



+ y- X] P(a;) J^(/(2/V)i^(x, y'; x, y) - p\y\x)v{x, y; x, y')) 
^ y,y' 

f p'{y'\x)v{x,y';x,y) v{x,y'; x,y)\ 

^ I X . ^ - log —, j7 . (24) 

V p'{y\x)v{x,y;x,y') v{x,y; x,y') J 

Before proceeding any further, notice that the formulas for U and S posses terms corre- 
sponding to the slow and fast dynamics subspaces. Moreover, the slow and fast dynamics 
terms in each equation have the same general structure. The same is true when each (slow 
or fast dynamics) term is compared with that on the right hand side of the corresponding 
equation in p^]. Finally, because of the adiabatic approximation, the fast-dynamics terms 
in U and S are equal, except for the multiplicative factor T. Hence, they cancel in U — TS 
and, in consequence, the time derivative for the free energy [F) is the same no matter wether 
a fast time scale exists or not [24 1 . 



B. Detailed balance 



So far, we have obtained all of our results without making use of the detailed balance 
condition. When the environment of a stochastic system is not driving it out of equilibrium, 
the system ultimately reaches an equilibrium steady state which is characterized by the 
fulfillment of detailed balance: 

P^ix, y)v{x, y; x', y') = p\x , y')v{x', y'; x, y). (25) 

Through the present section we denote the stationary distribution as p^{x,y)^ rather than 
p'^{x,y), to emphasize the fact that it obeys detailed balance and thus corresponds to ther- 
modynamic equilibrium. 
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Consider the effective transition probability defined in ([3]) and make use of (jl]) to arrive 
at the following expression: 

p{x)T{x;x') = ^p{x,y)v{x,y;x',y'). 

y,y' 

Assume now that the system is in equilibrium and substitute Eqn. fl25|) into the above 
equation to obtain 

p^{x)T{x; x') = p'{x')T{x'; x). (26) 

That is, Eq. fl26p is the form of the detailed balance condition for the variable with slow 
dynamics, with the probability distribution p'^{x) = '^yP'^{x,y). On the other hand for the 
fast dynamic variable, it follows from (jl]) that detailed balance implies that 

p''{y\x)v{x, y; x, y') = p''{y'\x)v{x, y'; x, y). (27) 

By employing the above results and following the procedure introduced by Ge and Qian 



10|, we can decompose U, S, and F as follows: 



with 



U = -Qd, F = -Ta, and S = a-^, (28) 



Qd = -if = ^ J2ipix')T{x'; x) - p{x)T{x- x')) log (29) 



A comparison of Eqns. ( |28l) and fl30l) with the equations defining the dissipation heat and 



the entropy production rate in [lOj, respectively, reveals that Qd and a posses the same 
mathematical structure as, and thus can be identified with those quantities. 

When the system is in equilibrium with detailed balance, U = F = S = 0. Furthermore, 
it is straightforward to verify that = o" = as well. We thus conclude from these results 
that the thermodynamic equilibrium state is characterized not only by the constancy in time 
of the thermodynamic state functions U, F, and S, but also by the existence of neither an 
energy flow nor an entropy production. 
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C. Process functions for systems without detailed balance 



We are now going back to Eqns. ( 122|) -( 12^ . We see by following the procedure in [lO| that, 
when detailed balance is not fulfilled, the entropy rate of change can still be decomposed as 



c Qd 

0=0- — — -, 
T ' 



where the entropy production rate is now given by 



p{x')T{x'; x) 
p{x)T{x; x') 



+ ^^v{x) ^{p%y'\x)v{x, y'\ x, y) - p\y\x)v{x, y\ x, y')) 



X log 



p''{y'\x)v{x,y'\x,y) 



p'{y\x)v{x,y;x,y') ' 
while the dissipated heat rate is 



Qa 



2 

x,x 



^^(p(x')T(x'; x) — p{x)T(x; x')) log 



T(x'; x) 
T(x; x') 



^pix) ^{p''{y'\x)v{x, y'; x, y) - p''{y\x)v{x, y; x, y')) 



X lot 



X y,y 
v{x,y';x,y) 



(31) 



(32) 



v[x,y;x,y') 

Observe that both a and Qa can be decomposed into two different terms with the same 
structure, each one of them corresponding to the slow and fast dynamics subspaces. 

Eqn. fl3T|l is one of the fundamental postulates of phenomenological irreversible thermo- 
dynamics (25|. Using these definitions we can also rewrite U and F as 



U = Qhk - Qd and F = Qhk - Ta, 



(33) 



where 



Q 



hk 



2 

x,x 



^^(]9(x')T(x'; x) — p{x)T{x; x')) log 



p*(x')T(x'; x) 
p'^{x)T{x; x') 



+ 



^p{x) ^{p\y'\x)v{x, y'\ x, y) - p\y\x)v{x, y\ x, y')) 



y,y 



X log 



p%y'\x)v{x,y';x,y) 
p'<{y\x)v{x,y;x,y') ' 
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This expression for Qhk can again be decomposed into two terms corresponding to the 
slow and fast dynamics subspaces. Observe that both terms have the same mathematical 
structure as the definition for the housekeeping heat in [lO]. Hence, we can identify Qhk 
with this quantity, originally introduced by Oono and Paniconi [lo[ [l^ and interpreted as 
the energy flow that has to be administered to the system to keep the stationary state out 
of equilibrium. 
Define now 

Mx,y\y) = ip'iy'\x)v{x,y';x,y) - p'{y\x)v{x,y;x,y')), 
B{x,y) = hgp'iylx), 

It is straightforward to verify that A is antisymmetric in y and y': A{x, y', y) = —A{x, y, y'). 
Moreover, since p'^{y\x) is by definition the stationary conditional probability distribution for 
variable y (conditioned to the value of x), it follows from Eqn. ([5]) that A{x, y', y) = 
Vx, y. Furthermore, as a function of y and y', A is an antisymmetric matrix with all its 
rows, thus all columns, summing zero. Then for any real vector B with component B{-,y): 

5^A(x,i/,i/') {B{x,y)-B{x,y')) 
y,y' 

= J2 B{x, y) [ ^(^' ^' y'n-H B{x, y') ( J2 A{x, y, y') J = 0. (34) 
y \ y' / y' \ y / 

This result further implies that 

ksTsr^, , , . , p'{x')T{x']x) 



Qhk = -|-^(]5(x')T(x';x) -p(x)T(x;x'))log: 



p'^{x)T(x; x') 
k T . 

+ ^p(x) Y{p\y'\x)v{x, y'; x, y) - p\y\x)v{x, y; x, y')) 



y,y 



v{x,y;x,y) 
v{x,y;x,y') 

a = — 2_](p(x')T(x';x) -p(x)T(x;x'))log 



2 ^ p(x)T(x;x') 

fee 
2 



+ ~^Yp(^) X^(p'(y'|a;)t;(x, y'; x, y) - p%y\x)v{x, y; x, y')). 



y,y 



v{x,y;x,y) 

x^og— -. 36) 

v{x,y;x,y') 
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Finally, the expression for U and S transform into 



U 




(37) 



S 



^{p{x')^{x'\ - p{x)i:{x\ x')) 




(38) 



while F remains the same as in Eqn. (1231) . 



Let us define 



Qfast 



^ ^p(a;) ^{p'{y'\x)v{x, y'\ x, y) - p\y\x)v[x, y\ x, y')) log 



v{x,y'\x,y) 
v{x,y;x,y') 



(39) 



We can see from this definition that Qfast is an energy flux related to fast time scale. 
Observe that Qfast appears as a summand in the expressions for Q^ fl32|) and Qhk (l35l) . 
while Qfast/T appears in the expression for a (!36|) . That is, the fast dynamics contributions 
to the dissipated heat, the housekeeping heat, and the entropy production rate are identical 
(except for a factor T in the case of a) in all three cases. Furthermore, Qfast cancels when 
Qd, Qhk, and Tcr are subtracted and this explains why such term does not appear in the 
expressions for f/, S, and F. 

D. Partial detailed balance with rapid pre-equilibrium 

Assume that detailed balance is fulfilled by the fast dynamics distribution {p{y\x)), but 
not necessarily by p{x). Then there is a rapid pre-equilibrium p{y\x) ~ p^(?/|a;), with p^(y|a;) 
satisfying Eqn. f l27|) . If this is the case, then Qfast = 0. Interestingly, the expressions for 
U, S, and F do not change. They are the same as in Eqns. ( I23l) . ( 1371) . and ( l38l) . except 
that p**(?/|x) is substituted by p^{y\x) whenever the former term appears. All this means 
that, having or not having detailed balance in the fast dynamics space makes a difference for 
the energy flows Qd and Qhk, as well as for the entropy production rate a (all of them are 
smaller in the first case because the contribution due to fast dynamics vanishes), however 
this difference is transparent to the rate of change of all thermodynamic state functions {U, 
S, and F). 
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E. Stationary distribution without detailed balance 

To analyze the behavior of the process variables when the system is at a steady state 
without detailed balance let us define 

A{x,x') = p'{x)T{x;x') -p'{'x)T{x';x), 
B{x) = \ogp^{x)^ 
C{x) = u'{x)/kBT. 

Clearly, A{x,x') is antisymmetric {A{x,x') = —A{x',x)). Moreover, since p'^{x) is the 
stationary probability distribution for variable x, it follows from Eqn. that A{x', x) = 
0. Hence, by the same result in Eq. flM|) we have 

= ^^(x',x)(C(a;')-C(x)) = 0. 

x,x' x,x' 

This last equation, together with fl25]) . ([STj), and (I55|) further imply that 

Ta = Q, = Qnk = ^ Y.^f ^x')T {x' ■ x) - f{x)T{x- x')) log + Qf^st. (40) 

x,x' ' 

when p{x) = p^{x) and p{y\x) = p^{y\x). Finally, it results from Eqn. (I40l) that U = S = 
F = in the stationary state. Indeed, we can see from flTTl) that F = in such case. 

The results in the foregoing paragraph corroborate the following: once the system reaches 
the steady state distribution, all the thermodynamic state functions (internal energy, free 
energy, and entropy) will remain constant. However, contrary to an equilibrium steady 
state in which detailed balance is fulfilled, a nonequilibrium steady state has nonzero fiuxes 
given by ( H0|) . The equalities between the fiuxes refiect both the energy conservation and 
the isothermal Clausius equality: on the one hand, to keep the system out-of-equilibrium, 
energy has to be supplied to the system (Qhk) which is then dissipated as heat (Qd)', while, 
on the other hand, entropy is produced in the process of the conversion of useful energy to 
heat (a = Qd/T). 

V. QUASI-STATIONARY AND STEADY- ADIABATIC PROCESSES 

The concepts of quasi-stationary and adiabatic processes are central to thermodynamics. 
In systems where the stationary state satisfies detailed balance, a quasi-stationary process 
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can be defined as a succession of states where a = 0, while an adiabatic processes is a 
succession of states satisfying Qd = 0. 

Oono and Paniconi jl?! generalized these concepts for systems with a non-equilibrium 
steady state (NESS) by defining the excess heat and the free energy dissipation rate as 
follows: 

Qex = Qd — Qhki (41) 

9 = Ta- Qhk, (42) 

and noting that, in terms of these variables, the rates S, U, and F can be rewritten as — see 
Eqns. (I5T]) and (155]): 

^ = ^^, U = -Qex, F = -9. (43) 

A comparison of Eqns. (|28]) and (jl3]) reveals that and a in systems where the sta- 
tionary state satisfies detailed balance can be respectively identified with Q^x and 9/T m 



NESS systems. Based on this identification Oono and Paniconi [17| generalized the con- 
cepts of quasi-stationary and steady-adiabatic processes for NESS systems as follows: a 
quasi- stationary process is a succession of states satisfying 6* = 0, while a steady-adiabatic 
process is a succession^ of states complying with Q^x = 0. After introducing these concepts, 
Oono and Paniconi [17j made extensive use of them in the development of their phenomeno- 
logical steady-state thermodynamics. Here we investigate how a time-scale separation affects 
these processes. 

After substituting ([52]), ([55]) and ([511) into and we obtain 

Qex = ^ 5^(j9(x')T(x'; x) - p{x)T{x- x')) log ^ (44) 
„ „/ p \x ) 



e = f E(P(^')T(a;'; x) - p{x)T{x; x')) log (45) 

Recall that the energy fiux related to the fast time scale Qfast — see Eqn. ([39]) — appears 
as a summand in Qd-, Qhk, and a. Hence, it cancels out at the time of subtracting these 
quantities and so it shows neither in Qex nor in 9. Consequently, 9 has no contribution 
whatsoever from the fast dynamics subspace. 

The fact that 9 depends only on the slow-dynamics subspace x means that the fast- 
dynamics subspace (y) does not infiuence whether a given process is quasi-stationary or not. 
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This result is in agreement with the adiabatic approximation we have made to reduce the 
system master equation, which is equivalent to assuming that the fast-dynamics subspace 
immediately equilibrates with the slow-dynamics state x. 

Regarding steady-adiabatic processes for NESS systems we see that, since Qex depends 
on the fast dynamics through sf{x) — see Eqns. (ITB]) and (1441) . the fast dynamics cannot 
be ignored while determining the adiabaticity of a given process. This can be more clearly 
appreciated by noticing that the following generalized Clausius equality is satisfied in a 
quasi- stationary process: 

Q Qex 



VI. CONCLUDING REMARKS 



In this work we have extended the results of a recent paper 10| where a rather complete 
thermodynamic formalism was introduced for discrete-state, continuous-time Markov pro- 
cesses with and without detailed balance. Our main objective was to investigate whether 
the thermodynamic structure is invariant in a multiscale stochastic system. By invariance 
we mean that the relation between state and process variables remains unchanged when the 
system is viewed at different time scales. 

We proceeded as follows. First, we assumed that the states of a system can be classified 
according to the propensities of the transitions among them. More precisely, we supposed 
that every state can be represented by a dual vector {x,y), and that transitions involving 
changes in y alone are much more probable than those involving changes in x or in both x and 
y. Then, we imposed an adiabatic approximation to deduce a reduced master equation for 
the slower time scale. Finally, we analyzed the implications of this adiabatic approximation 
on the thermodynamic formalism introduced by Ge and Qian jlo| . 

As it resulted, all thermodynamic variables and their time derivatives can be separated in 
a very natural way into contributions from the slow and fast time scales. The only exceptions 
being the Helmholtz free energy and its time derivatives, which only involve terms due to 
the slower time scale. In other words, the Helmholtz free energy (which characterizes the 
system spontaneous organization) is invariant with and without a fast time scale. This 
happens because, having reached its stationarity, the fast time-scale probability distribution 
does not contribute to the free energy. The same cannot be said about the entropy and the 
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internal energy, both of which have fast dynamics contributions. 

The above discussed results are important because they provide a framework to study 
the thermodynamics of complex Markov processes where time-scale separation is possible. 
Some examples where this framework can be useful may be enzymatic reactions in which 
one of the chemical steps is much slower than the rest, of gene regulatory networks where, 
typically, transcription initiation is an infrequent process, as compared for instance with 
translation initiation and post-translational modification of the resulting proteins. 

On the other hand, the same results can also be interpreted from a more fundamental 
thermodynamic perspective. We elaborate on these ideas next. The dynamics on a fast time 
scale contribute an entropic term to the energy function Us{x), for the slow dynamics. This 
fact allows to identify us{x) as a free energy. Indeed, us{x) results to be the conditional free 
energy; a concept extensively used in equilibrium thermodynamics as one does not usually 
work with pure mechanical energy, but with a coarse grained conditional free energy, and 
develop a partition function thereafter. 

On the other hand, entropy-enthalpy compensation has been extensively studied in bio- 
chemistry 26|, 1271]. The strong form of this phenomenon occurs when variations in AH 
and AS, caused by regular changes in some experimental variable (excluding temperature), 
exhibit a linear correlation. In this case AG will be small relative to the range of values 
expected from the experiment. 

As pointed out by Qian 2J] , internal energy is the equivalent of enthalpy, while Helmholtz 
free energy is the equivalent of Gibbs free energy, in the type of systems here studied. 
In that respect, entropy-internal energy compensation in these systems is tantamount to 
entropy-enthalpy compensation in isobaric ones; and the effect of entropy-internal energy 
compensation will be small F changes. The insight from the present work is that the 
compensating part of entropy and internal energy is the contribution from fast dynamics; 
i.e., rapid fluctuations. 

The expressions we derived for dU /dt and dS/dt contain terms associated to the slow and 
fast time scales. However, when the adiabatic approximation is imposed, the contributions 
from the faster time scale become equal, except for a factor T in dS/dt, see Eqns. fl22|) and 
As a consequence, the expression for dF/dt only includes a slow dynamics term — 
see Eqn. fl23|) . That is, we have entropy-internal energy cancelation for the fast dynamics 
contributions. As a matter of fact, it is impossible to know whether a fast time scale exists 
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or not from the perspective of the Helmholtz free energy. These results are in complete 
agreement with previous studies which prove the existence of entropy-enthalpy compensation 
by considering that, in response to a small perturbation, the free energy change of an 
stationary system is independent of the system thermodynamic environment, while the 
entropy and the internal energy changes depend on the environmental constraints 2^, 
Recall that the adiabatic approximation is equivalent to assuming that the fast dynamics 
distribution p{y\x) reaches its stationary value instantaneously for every state x{t) 
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Appendix A: Rate of change of the thermodynamic state variables 



After differentiating Eqns. (fTOj) . (JH]), and (fT7|) we obtain the following results for the 
time derivatives of the internal energy: 



U = y^^p{x){us{x) + uf{x)) + ^^p{x)uf{x), 

X X 

k j^T^ - ^ (xj 
= y](p(^')'^(a;'; - p{x)T{x; x')) log — — 



x,x' 



2_]p(a;) 2_](p(2/'|a;)t'(a;, y'; x, y) - p{y\x)v{x, y; x, y')) log 



2 ^ ^ v^{y'\x) 

X y,y' f \a \ J 



of the free energy: 



X) 



ksT sr^ , , , p{x')p''{x) 



^^(p(a;')T(a;'; a;) — p{x)T{x; x')) log 



2 ' ' p(x)p'^{x') 



x,x 



p{y\x)p'{y'\x 
and of the entropy: 



^ y,y' 

p{y'\x)p'{y\x) 
X log ■ 



S = ^^pix) (knlogpix) + sf{x)) + ^p{x)sFix), 



X 

k 



B 



2 



^^(p(x')T(x'; x) — p{x)T(x; x')) 



p{x')T{x';x) T(x'; x] 



p(a;)T(x;x') T(x;x') 
+~?~ X^(^'(2/V)t'(a;, y'; x, y) - p{y\x)v{x, y; x, y')) 



y,y 



p{y'\x)v{x,y';x,y) v{x,y';x,y) 
^ log /IN/ ^ - log 



p{y\x)v{x,y;x,y') v{x,y;x,y') 
In the derivation of the previous equations we have taken into consideration that 

^p(x)uf(x), ^p(x)/f(x), ^p(x)sf(x) ^ 0. 

XXX 

The demonstration of these last relations is straightforward and follows from the fact that 
z/(x, y, x', y') ~ for all x 7^ x', which is the basic assumption underlying the time scale 
separation. 
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